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ABSTRACT 

Historical  accounts  of  military  deception  abound,  but  there  are  few  historical 
accounts  of  counter-deception,  and  fewer  operational  theories.  This  paper  describes  a 
business  process  and  semi-automated  tools  for  detecting  deception.  The  counter¬ 
deception  business  process  begins  with  hypothesis  generation.  This  consists  of  automated 
course  of  action  generation  in  tactical  situations;  strategic  situations  require  hypothesis 
elicitation  from  analysts.  Next,  hypotheses  and  related  potential  evidence  are  represented 
by  a  Bayesian  belief  network.  This  network  is  the  basis  of  a  diagnostic  analysis  derived 
from  classification  theory.  The  result  is  a  weighted  list  of  possible  observations  that:  (1) 
identify  distinguishing  evidence  that  a  deceiver  must  hide  and  a  counter-deceiver  must 
uncover,  (2)  isolate  local  deception  in  intelligence  reporting  and  sensing  from  global 
deception,  and  (3)  identify  circumstances  when  it  might  be  fruitful  to  entertain  additional 
hypotheses.  We  illustrate  this  process  by  describing  how  it  could  have  been  used  by  the 
Japanese  Navy  before  the  Battle  of  Midway  to  detect  the  American  denial  and  deception 
tactics. 

INTRODUCTION 

This  paper  considers  counter-deception  from  a  psychological,  rather  than  cultural 
perspective.  First,  we  summarize  the  cognitive  aspects  of  counter-deception.  Next,  we 
describe  a  process  developed  in  the  intelligence  community  called  the  Analysis  of 
Competing  Hypotheses  (ACH).  We  describe  how  we  correct  ACH  to  account  for 
cognitive  factors  that  make  people  poor  at  detecting  deception.  We  call  this  modified 
process  ACH-CD.  Then  we  describe  semi-automated  tools  that  demonstrate  that  ACH- 
CD  is  sufficient  for  counter-deception.  Finally,  we  demonstrate  how  the  modified  process 
provides  a  basis  for  military  counter-deception  with  a  demonstration  and  descriptive 
application  to  the  battle  of  Midway,  from  Japan’s  perspective. 

Why  is  Counter-deception  hard? 

In  this  paper,  the  term  counter-deception  means  detecting  or  recognizing  a 
deception.  Note  that  successful  counter-deception  does  not  necessarily  imply  that  one 
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knows  the  adversary’s  true  course  of  action.  We  use  the  term  deception  to  include  denial, 
or  hiding,  which  we  consider  a  component  of  deception  behavior,  and  deceptive 
misleading  or  dissimulation. 

Put  simply,  people  are  deceived  because  they  do  not  systematically  consider 
alternative  explanations  for  the  evidence  they  observe  (Johnson,  et  al  2001,  Heuer  1981, 
Heuer  1999,  Whaley  and  Busby  2002)  and  incorrectly  weigh  the  evidence  they  do  have 
(Dawes  2001).  These  behaviors  occur  because  of  memory  limitations  and  related 
reasoning  heuristics  that  evolved  to  deal  with  a  high  base  rate  world  (Gilovich  et  al. 

2002).  The  result  is  that  people  often  dismiss  important  evidence,  prematurely  prune 
alternative  hypotheses,  and  jump  to  conclusions.  These  make  people  and  organizations 
easy  to  deceive. 

Johnson  and  his  associates  (2001)  note  that  human  evidential  reasoning  is  mainly 
adequate  for  frequently  experienced  events.  Reasoning  heuristics  that  evolved  to  be 
cognitively  efficient  and  effective  in  our  high-base  rate  world  often  result  in  biased 
reasoning  -  grossly  over  or  under  estimating  probabilities  -  when  one  is  faced  with  low- 
base  rate  events  such  as  deception  (Gigerenzer  et  al  1999).  Since  deception  is  relatively 
rare,  it  is  not  surprising  that  people  are  poor  at  counter-deception.  Heuristics  can  result  in 
the  following  analytic  errors  that  hinder  effective  counter-deception: 

•  Poor  anomaly  detection:  Analysts  miss  environmental  cues  of  anomalies,  or 
prematurely  dismiss  anomalies  as  irrelevant  or  inconsistent  with  other  intelligence. 
(We  do  not  intend  to  imply  that  counter-deception  is  mainly  a  process  of  anomaly 
detection,  in  the  statistical  sense.  We  use  the  term  anomaly  to  denote  evidence  that  is 
not  consistent  with  the  analyst’s  current  beliefs  or  expectations  about  the  state  of  the 
world  or  the  predicted  actions  of  an  adversary); 

•  Misattribution:  inconsistent  or  anomalous  events  are  often  attributed  to  collection 
gaps  or  processing  errors,  rather  than  to  deception  tactics; 

•  Failure  to  link  denial  and  deception  tactics  to  deception  hypotheses:  When  they  do 
notice  anomalies,  analysts  often  fail  to  recognize  anomalous  evidence  as  indicators  of 
denial  and  deception  tactics,  possibly  suggesting  strategic  deception  objectives; 

•  Inadequate  support  for  deception  hypotheses:  Analysts  fail  to  link  their  assessment  of 
an  adversary’s  deception  tactics  to  the  adversary’s  strategic  goals;  i.e.,  analysts  fail  to 
test  denial  or  deception  courses  of  action  (COAs)  against  all  the  available  evidence. 

Counter-Deception  Business  Process  Model 

Johnson  et  al  (2001)  conducted  a  cognitive  task  analysis  of  forensic  accountants 
performing  counter-deception  auditing  analysis  with  varying  degrees  of  success.  Johnson 
et  al  developed  a  four-part  analysis  business  process  to  describe  and  address  the  four 
problematic  heuristics  summarized  above.  In  this  paper  we  describe  how  we  adapted  and 
augmented  the  “Analysis  of  Competing  Hypotheses”  technique  for  deception  detection  to 
address  one  key  part  of  this  business  process.  We  are  also  identifying  and  adapting  useful 
theories  or  algorithms  for  the  other  components  of  the  Johnson  et  al  counter-deception 
business  process  model. 
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The  other  counter-deception  theories  we  applied  to  the  Johnson  et  al  counter-deception 

business  process  model  are: 

•  Whaley  &  Busby’s  (2002)  “Congruity  Theory  &  Ombudsman  Method”  explicitly 
address  the  problem  of  Poor  anomaly  detection  by  identifying  data  collection 
techniques  likely  to  surface  anomalies  related  to  denial  or  deception  tactics. 

•  R.  V.  Jones’s  (1978, 1989,  1993)  “Theory  of  Spoof  Unmasking”  can  be  adapted  to 
address  the  problem  of  Mis  attribution  >  i.e.,  to  avoid  attributing  inconsistent  or 
anomalous  events  to  collection  gaps  or  processing  errors  rather  than  to  denial  or 
deception  tactics.  Jones  advocates  analysis  of  anomalies  through  the  use  of  multiple 
channels  of  intelligence  (e.g.,  signal  intelligence-SIGINT,  imagery  intelligence- 
IMINT,  open  source  intelligence — OSINT)  applied  to  anomalies,  examined  at  various 
resolutions  (both  higher  and  lower).  We  would  add  tat  analysts  should  compare  not 
only  the  expected  means  of  these  observations  to  base-rate  data,  but  also  the 
anomalous  data’s  variance  and  skewedness,  since  anomalies  whose  averages  seem 
normal  may  be  revealed  as  deceptive  because  the  data  do  not  vary,  or  vary  in  the 
normal  directions,  as  do  base-rate  data.  Finally,  Jones  advises  “natural”  or  planned 
operational  “experiments”  that  force  the  enemy  to  provide  additional  intelligence  that 
will  highlight  possible  denial  or  deception  tactics.  At  the  Battle  of  Midway,  the 
Americans  used  an  operational  experiment  to  confirm  that  the  main  Japanese  target 
was  Midway  Island  and  not  one  of  the  several  Japanese  diversionary  targets.  Midway 
defenders  signaled  Hawaii  in  the  clear  that  the  island’s  water  purification  system  was 
off-line.  Japanese  SIGINT  picked  up  the  bait  and  alerted  Tokyo  that  “AF,”  codename 
of  the  main  target  of  the  Japanese  operation,  was  short  of  water,  thus  linking  the 
Midway  bait  to  the  Japanese  plans  that  American  SIGINT  analysts  had  already 
intercepted  and  reconstructed. 

•  We  adapted  Heuer’s  “Analysis  of  Competing  Hypotheses”  (ACH,  Heuer  1999) 
technique  for  counter-deception  analysis  to  address  the  Failure  to  link  denial  and 
deception  tactics  to  deception  hypotheses.  This  adaptation  assesses  the  likelihood  of 
indicator  events  and  evidence  across  probable  COAs,  including  denial  and  deception 
and  separately  assesses  anomalies  due  to  sensors,  collection,  or  processing  errors  and 
anomalies  due  to  denial  or  deception,  i.e.,  the  issue  of  “local  versus  global  deception.” 

•  Finally,  Johnson  et  al’s  (2001)  “Cognitive  Model  of  Fraud  and  Deception  Detection” 
organizes  these  four  parts  into  a  counter-deception  business  process  (Figure  1)  that 
specifically  addresses  the  problem  of  Inadequate  support  for  deception  hypotheses. 
Johnson  et  al  appreciate  the  need  to  address  the  sensitivities  of  hypotheses  (or  COAs) 
to  the  indicators  of  denial  and  deception  tactics  (anomalous  evidence)  in  order  to 
effectively  focus  intelligence  collections  and  operational  intelligence  experiments,  to 
confirm  suspicions  of  deception,  and  to  reduce  the  uncertainties  of  deception 
estimates.  They  also  note  the  importance  of  the  intelligence  analyst  making  an 
adequate  case  for  deception  detection.  Often  deceptions  are  constructed  on  a 
foundation  of  the  preconceptions  of  the  analyst’s  customers,  who  will  be  reluctant  to 
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forsake  these  preconceptions  without  a  strong  case  being  made  by  the  analyst  for  the 
deception  hypotheses. 


What  is  an  analyst  to  do? 

Anyone  who  reads  the  newspapers  knows  that  deception  plagues  the  intelligence 
community.  Heuer  (1999)  developed  a  protocol  called  Analysis  of  Competing 
Hypotheses  (ACH)  in  part  to  address  analysts’  susceptibility  to  deception.  ACH  consists 
of  the  following  steps  (simplified  and  slightly  reordered;  see  Heuer  1999  for  the  original 
eight-step  formulation): 

1 .  Prepare  a  matrix  listing  hypotheses  vs.  evidence. 

a.  Identify  the  possible  hypotheses  to  be  considered. 

b.  List  significant  observed  evidence  and  assumptions  for  and  against  each 
hypothesis. 

2.  Draw  tentative  conclusions  about  the  relative  likelihood  of  each  hypothesis,  based 
on  the  evidence,  focusing  on  evidence  which  is  inconsistent  with  hypotheses. 

3.  Analyze  sensitivity  of  the  conclusion  to  a  few  critical  items  of  evidence. 

4.  Report  conclusions. 

5.  Identify  future  observation  that  may  indicate  events  are  taking  a  different  course 
than  expected. 

ACH  helps  analysts  to  compare  evidence,  arguments,  and  assumptions  (e.g., 
intelligence)  to  possible  hypotheses  (estimates  of  the  situation  or  courses  of  actions). 
ACH  ensures  alternative  hypotheses  are  considered  equally  and  fully  and  that  the 
information  value  of  the  evidence  and  assumptions  is  applied  to  the  hypotheses. 
Structuring  helps  analysts  probe  and  challenge  evidence  and  assumptions  and  test  the 
support  for  the  hypotheses.  Heuer  wrote: 
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“Simultaneous  evaluation  of  multiple,  competing  hypotheses  . . .  takes  far  greater 
mental  agility  than  listing  evidence  supporting  a  single  hypothesis  that  was  pre¬ 
judged  as  the  most  likely  answer.  It  can  be  accomplished,  though,  with  the  help  of 
the  simple  procedures. .  ..The  ACH  procedure  has  the. .  .advantage  of  focusing 
attention  on  the  few  items  of  critical  evidence  that  cause  the  uncertainty  or  which, 
if  they  were  available,  would  alleviate  it.  This  can  guide  future  collection, 
research,  and  analysis  to  resolve  the  uncertainty  and  produce  a  more  accurate 
judgment.” 

Lately,  several  U.S.  intelligence  agencies  have  advocated  using  ACH  to  enhance 
intelligence  analysis  and  estimation.  For  example,  the  Central  Intelligence  Agency 
offers  its  analysts  workshops  on  “Alternative  Analysis”  methods  (Directorate  of 
Intelligence  2002).  The  CIA  re-published  Heuer’s  The  Psychology  of  Intelligence 
Analysis  in  1999  and  posted  Heuer’s  book  on  the  CIA’s  website.  Morgan  Jones,  who 
learned  ACH  as  a  CIA  analyst,  featured  the  technique  prominently  in  his  book,  The 
Thinker ’s  Toolkit:  Fourteen  powerful  techniques  for  problem  solving  (Jones  1 998),  which 
is  also  cited  widely  in  U.S.  intelligence  agency  analyst  training. 

Adapting  ACH for  Counter-Deception 

Our  concern  is  that  ACH  could  lead  analysts  to  be  more  susceptible  to  deception. 
In  particular,  step  2  suggests  weighing  hypotheses  in  light  of  evidence  (p(Hi|E),  an 
heuristic  Dawes  (2001)  notes  as  responsible  for  many  of  the  reasoning  errors  he  dubs 
“everyday  irrationality.”  The  problem  with  step  2  is  that  p( Hi|E)  neglects  the  base  rates 
of  the  evidence,  p(E),  and  the  hypothesis,  p( Hi).  The  prior  probability  of  H,  being  mainly 
subjective,  can  be  a  source  of  bias. 

Neglect  of  base  rates  features  prominently  in  many  writings  on  evidential 
reasoning  errors  (e.g.,  Bums  2004a,  b,  c),  for  example,  foster  the  confirmation  bias  if 
analysts  observe  evidence  that  is  consistent  with  H,  even  when  the  evidence  might  be 
consistent  with  alternate  hypotheses.  The  confirmation  bias  results  in p(H|E)  being 
greater  than  p( H),  which  may  already  reflect  bias.  But  more  important  for  counter¬ 
deception  reasoning,  assessing  (p(Hi|E)  fails  to  direct  the  analyst’s  attention  to  the  false 
positive  rate  of  the  evidence,  p(E\  not  Hi). 

People  are  much  more  susceptible  to  deception  if  they  do  not  normatively  account 
for  both  p( E|  Hi)  and  p( E|  not  Hi).  They  allow  a  deceiver  to  simulate  evidence  that  is 
often  associated  with,  but  not  necessary  for  a  particular  course  of  action  (hypothesis). 
They  also  are  prone  to  the  confirmation  bias  if  they  observe  things  that  are  consistent 
with  more  than  one  hypothesis. 

To  illustrate,  consider  a  hypothetical  strategic  deception.  Say  intelligence 
collection  detects  Krypton  gas  in  some  Middle  Eastern  country.  This  suggests  the 
conclusion  that  the  target  country  may  have  an  active  nuclear  weapons  program,  since 
Krypton  gas  is  a  by-product  of  uranium  enrichment.  The  argument  might  be  summarized 
this  way: 


Midway  Revisited 


©2004  The  MITRE  Corporation 


5 


Detect  Krypton 

/^enrichment  |  Krypton)  =  high  4  /?  (enrichment  program)  =  high 
Leading  to: 

p  (nuclear  program)  =  high 
Intuitively  appealing,  but  wrong. 

The  two  involved  errors  are  hard  to  avoid,  due  to  our  experience  in  high  base  rate 
worlds:  first,  generating  hypotheses  based  on  evidence  “usually  works”  in  our  causally 
organized  experience,  and  second,  p(Ej|Hi)  often  provides  a  good  approximation  for 
p(Hi|Ej).  ACH  does  not  warn  the  analyst  about  either  of  these  errors.  But  the  error  most 
difficult  to  correct  is  failing  to  consider  p(Ej|not  Hi). 


In  our  example, 
p( Ej|  not  Hi)  is  the 
probability  of  detecting 
Krypton  gas  when  there 
is  no  enrichment 
activity.  Depending  on 
the  situation,  this 
probability  might  not  be 
negligible.  One 
common  use  of  Krypton 
gas  is  insulation  in 
double-pane  windows. 

Another  use  is  to  test 
pipelines  for  leaks.  If  pipelines  are  common  in  the  target  country,  then: 

pfKrypton  |  not  enrichment)  =  medium  to  high 

We  modeled  this  example  in  a  Bayesian  belief  network  (BBN),  first  by  following 
the  guidance  of  Jones  (1998)  and  the  Directorate  of  Intelligence  (1987),  Deception 
Analysis  Methodology,  as  shown  in  Figure  2.  Applying  Bayes’  rule  with  some 
representative  numbers  might  lead  one  to  determine  that  the  detection  of  Krypton  gas  is 
not  the  definitive  evidence  one  thought.  A  possible  deception  hypothesis  is  that  the  target 
might  simulate  a  weapons-grade  enrichment  program,  perhaps  as  an  inexpensive 
deterrent  to  neighbors. 


Figure  2:  Bayesian  belief  network  using  p(H|E) 
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Observing  Krypton  in  the  first  model  (Figure  1)  gives  a  high  likelihood  that  there 
is  a  nuclear  enrichment  program,  “Nuke  program”  (7:3  odds  in  favor  of  nuclear 
enrichment).  A  possible  deception  hypothesis,  that  that  the  target  country  might  simulate 
a  weapons-grade  enrichment  program  by  the  intentional  releasing  Krypton,  perhaps  as  an 
inexpensive  deterrent  to 
neighbors,  is  also  shown, 

“Bluff!”.1  This  model  leads 
to  a  very  low  estimate  of 
the  probability  of  this 
possible  deception 
(roughly  4  to  1  odds 
against  deception).  Notice 
also  that  the  observation  of 
Krypton  leads  to  a  very 
high  likelihood  of  pipeline 
testing  (“Testing  1”  19:1 
odds  in  favor). 


Modeling  the  same 

ACH  situation  following  Bayes’s  rule,  i.e.,/?(E|H),  and  the  same  representative  numbers 
(Figure  3)  leads  one  to  determine  that  the  detection  of  Krypton  gas  is  not  a  definitive 
indicator  of  uranium  enrichment  as  suggested  by  the  model  in  Figure  2.  In  the  Bayes- 
based  ACH  BBN  model,  the  observation  of  Krypton  could  indicate  either  nuclear 
enrichment  or  pipeline  testing  (roughly  50:50  odds),  and  the  probability  of  the  possible 
deception  is  considerably  higher  (2  to  1  odds  against  deception)  than  in  the  first  ACH 
model  (Figure  2,  4  to  1  odds  against). 

That  is,  by  asking  the  Bayesian  question,  what  is  p( E|H);  i.e.,  how  likely  am  I  to 
detect  Krypton,  given  the  three  hypothetical  causes:  enrichment,  bluffing,  or  pipeline 
testing;  the  analyst  is  less  likely  to  over-estimate,  or  under-estimate,  the  likelihood  of  the 
hypotheses,  compared  to  a  p(R\E)  implementation  of  ACH."  In  short,  counter-deception 
analysts  using  ACH  should  consider  the  evidence  and  hypotheses  following  the  Bayesian 
normative  prescription,  /?(E|H),  rather  than  the  intuitive  p( H|E)  recommendation  often 
found  in  the  interpreters  of  Heuer’s  (1999)  ACH  method.. 

Summarizing,  ACH  offers  a  promising  technique  for  counter-deception  analysis, 
but  some  modification  is  needed  so  that  hypothesis  generation  includes  appropriate  denial 
and  deception  COAs,  and  the  ACH  is  used  to  elicit  or  estimate  both  p(Ej|Hi)  and  p( Ej|  not 
Hi).  We  call  our  process,  which  is  ACH  with  these  modifications,  “ACH-CD.”  The  next 
section  sketches  how  we  partially  automate  ACH-CD  for  counter-deception  decision 
support. 

ACH-CD 

ACH-CD  extends  Heuer’s  ACH  process  to  do  two  things  that  are  likely  to  be 
missed  if  one  naively  follows  ACH.  First,  it  incorporates  p( E|  not  H)  in  elicitation  and 
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computation  of  the  diagnostic  impact  of  evidence.  Second,  it  helps  analysts  consider 
deception  hypotheses. 

It  is  useful  to  distinguish  two  situations  where  counter-deception  might  be 
required.  The  first  we  call  tactical  counter-deception.  Tactical  situations  are  those  where 
the  adversary’s  goal  is  fairly  obvious  and  the  issue  is  detecting  how  he  plans  to 
accomplish  that  goal.  Military  operations  and  magic  acts  are  examples  of  tactical  counter¬ 
deception.  We  consider  a  situation  one  of  strategic  counter-deception  when  the  issue  at 
hand  is  what  the  adversary  wishes  to  accomplish.  Often  in  strategic  situations,  the 
adversary’s  course  of  action  is  unobservable  or  largely  irrelevant  to  the  main  analysis. 

Automating  ACH-CD 

We  have  developed  two  computer  programs  that  partially  automate  ACH-CD.  One 
technique  (Elsasser  and  Stech  2003),  is  focused  on  tactical  situations.  It  involves  the 
following  steps: 

1 .  For  a  given  situation,  state  one  or  more  possible  goal  states. 

2.  Automatically  generate  hypotheses,  in  the  form  of  state-based  plans  (courses  of 
action),  that  can  accomplish  the  goal  state(s). 

3.  Automatically  convert  the  course(s)  of  action  (usually  a  contingency  plan)  to  a 
Bayesian  belief  network. 

4.  Perform  sensitivity  analysis  on  the  network  by  sequentially  choosing  possible 
outcome  states  and  computing  a  factor  of  the  optimal  Bayesian  dichotomizer  (we 
omit  the  prior  probability  terms,  as  they  are  unnecessary  and  likely  to  be  biased): 

jp(Ei|H)//>(Ei|  not  H),  for  all  states  Ei 

The  log  of  the  state  odds  ratios  provides  an  indicator  of  the  impact  of  each  state 
on  the  probability  of  an  outcome  (hypothesis)  of  interest,  ordered  by  time  of  potential 
observation.  This  yields  three  sets  of  time-state  intervals,  representing:  the  states  that 
must  be  hidden  (denied),  observations  that  have  no  probative  value  to  the  observer,  and 
the  states  that  one  might  simulate  to  deceive  an  adversary.  Importantly,  the  list  often  will 
include  negative  information  -  states  that  are  highly  informative  if  one  does  NOT 
observe  them.  Reasoning  about  negative  evidence  is  particularly  problematic  for  people. 

The  counter-deception  analyst  can  use  these  results  to  direct  collection  to  the  most 
probative  evidence.  By  knowing  which  states  (observations)  have  no  probative  value,  one 
can  avoid  the  confirmation  bias,  basically  seeing  everything  as  support  for  one’s 
preconceived  notions  about  the  adversary’s  intent.  A  deception  planner  can  use  the 
results  of  this  system  to  determine  which  observations  must  be  denied  or  have  a  plausible 
cover  story.  Strong  indicators  of  an  alternative  hypothesis  might  be  simulated  to  mislead 
the  adversary. 

Our  second  process  is  aimed  at  strategic  deceptions  that  do  not  have  a  strong 
course  of  action  analysis  component,  such  as  the  question  “does  Iran  have  a  nuclear 
weapons  program?”  This  process  starts  with  the  elicitation  of  a  set  of  mutually  exclusive 
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hypotheses,  stated  as  True/False  propositions.  We  automatically  include  a  non- 
informative  hypotheses,  labeled  “Other,”  so  the  list  is  exhaustive.  This  is  followed  by 
eliciting  the  items  of  evidence  that  are  available.  The  most  time  consuming  step  is 
eliciting  from  the  analyst(s)  estimates  of  p(Ei|Hj)  and  p( Ei|  not  Hj)  for  all  evidence  Ei  and 
hypothesis  Hi.  In  practice,  many  of  the  elements  of  evidence  will  be  uninformative  (50- 
50)  for  some  of  the  hypotheses. 

After  the  elicitation,  a  Bayesian  belief  network  is  created  and  the  sensitivity 
analysis  process  described  above  is  performed.  The  result  is  a  list  of  evidence  states  that, 
if  observed,  would  have  the  most  significant  impact  on  the  likelihood  of  a  given 
hypothesis. 

The  remainder  of  the  paper  describes  how  the  Japanese  Navy  might  have  used  the 
first  of  these  methods,  the  ACH  and  ACH-CD  counter-deception  evidence  elicitation  and 
evaluation  procedures,  to  improve  their  counter-deception  assessments  of  American 
courses  of  action  prior  to  the  Battle  of  Midway. 

APPLYING  ACH  TO  MILITARY  COUNTER-DECEPTION 

We  illustrate  how  ACH-CD  can  address  military  counter-deception  by  examining  the 
Japanese  planning  for  the  Battle  of  Midway  from  the  viewpoint  of  the  Imperial  Japanese 
Navy  (JN).  In  planning  for  this  operation  the  JN  conducted  war  plan  reviews,  table-top 
exercises,  and  naval  exercises  which  shared  the  same  steps  as  Heuer’s  ACH,  but  as 
disjointed  planning  episodes,  not  as  an  integrated  analytic  process,  as  Heuer  advocates. 
Many  of  the  tactical  and  operational  problems  that  contributed  to  the  defeat  of  the  JN  at 
Midway  were  specifically  identified  and  discussed  during  these  JN  ACH-like  exercises, 
but  the  Japanese  planners  ultimately  dismissed  the  identified  problems,  or  met  them  with 
inadequate  half-measures  or  inappropriate  ripostes.  As  JN  planning  assumptions  were 
shown  to  be  incorrect  during  the  operation  itself,  the  JN  failed  to  replan  or  adjust  its 
operations. 

JN  Planning  Hypotheses  (explicitly  surfaced  by  the  JN  before  the  battle  in  ACH-like 
exercises): 

•  HI:  U.S.  Pacific  Fleet  would  respond  to  the  JN  invasion  of  Midway,  sending  its 
remaining  aircraft  carriers  (CVs)  to  attempt  to  retake  Midway. 

•  H2\  U.S.  Pacific  Fleet  would  not  respond  to  the  JN  invasion  of  Midway;  letting  Japan 
extend  its  naval  base  perimeter  to  mid-Pacific. 

•  H3:  U.S.  Pacific  Fleet  aircraft  carriers  will  be  waiting  near  Midway  to  attack  the  JN 
Carrier  Battle  Group  (Kido  Butai). 

The  JN  received  considerable  intelligence  and  had  operational  and  tactical  experience 
before  the  Battle  of  Midway  that  was  relevant  to  assessing  these  three  hypotheses  (see 
Appendix  1  and  sources).  While  historical  accounts  strongly  indicate  that  the  JN  surfaced 
these  hypotheses  explicitly  in  its  ACH-like  exercises,  the  record  is  clear  that  the  JN  never 
made  an  integrated  attempt  to  consider  all  the  evidence  and  all  of  the  hypotheses  as 
advocated  by  Heuer’s  ACH  method.  That  is,  the  hypotheses  were  raised  and  considered 
episodically  and  evidence  was  dismissed  piece-meal.  The  JN  planners  never  considered 
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all  the  evidence  against  these  hypotheses  as  they  assessed  the  strengths  and  weaknesses 
of  their  planning  assumptions.  Much  of  the  evidence  reflecting  JN  tactical  and 
operational  weaknesses  (e.g.,  in  intelligence,  reconnaissance  and  surveillance,  ISR)  was 
largely  ignored  in  the  design  of  “Operation  MI.”  In  short,  the  JN  neither  organized  an 
assessment  of  all  the  intelligence  and  evidence  in  planning  Operation  MI,  nor 
acknowledged  the  impact  of  important  evidence  that  reflected  potential  problems  in 
executing  the  plan. 

Had  the  JN  simply  drawn  tentative  conclusions  using  Heuer’s  ACH  technique  (see 
Table  1,  top),  that  is,  assessed  the  relative  likelihood  of  each  hypothesis  based  on  the 
evidence  they  had  available, />(Hj|Ej),  the  JN  might  have  estimated  that  it  was  perhaps  as 
likely  the  U.S.  Pacific  Fleet  carriers  could  be  waiting  to  ambush  the  Kido  Butai  (H3),  or 
that  the  U.S.  carriers  would  not  respond  as  the  JN  intended  to  Operation  MI  (H2),  as  was 
the  hypothesis  that  the  U.S.  would  respond  as  the  JN  assumed  to  the  invasion  of  Midway 
(HI). 

Had  the  JN  assessed  the  evidence  using  the  ACH-CD  manner  we  recommend  (see 
Table  1,  bottom),  that  is,  assessing  the  relative  likelihood  that  the  evidence  would  be 
observed  given  each  of  the  hypotheses  (p(Ej|H,)  and  p( Ej|  not  Hj)),  the  JN  might  have 
concluded  that  H2  and H3  were  as  likely  as  HI,  and  that  the  available  evidence  did  not 
strongly  support  the  JN  favored  CO  A,  HI. 


Table  1:  Evaluating  JN  Hj 

/potheses  j 

Likelihoods 

ACH:  Evaluating  JN  Hypotheses  against  Evi 
hypotheses  in  light  of  evidence):  p(Hj|Ej) 

dence  (likelihood  of 

See  Appendix  2] 

HI:  U.S.  will  respond 
to  JN  invasion  of 
Midway 

H2 :  U.S.  will  not 
respond  to  JN 
invasion  of  Midway 

H3:  U.S.  will  be 
waiting  near  Midway 

Evidence  Items  In  Favor 

9 

7 

10 

Evidence  Items  Opposed 

9 

6 

7 

Evidence  Items  Uncertain 

0 

5 

1 

Likelihoods 

ACH-CD:  Evaluating  JN  Evidence  against  Hypotheses  (likelihood  of 
evidence  in  light  of  hypotheses):  p(Ej|H|)  &  p( Ej|  not  Hi)  [See  Appendix  31 

Evidence  Items  In  Favor 

8 

9 

9 

Evidence  Items  Opposed 

9 

8 

7 

Evidence  Items  Uncertain 

1 

1 

2 

In  other  words,  the  JN  might  have  done  much  to  overcome  the  first  two  major 
hurdles  impairing  their  counter-deception  analysis:  Poor  anomaly  detection  (missing 
anomalies  or  prematurely  dismissing  anomalies  as  irrelevant  or  inconsistent)  and 
Misattribution  (attributing  inconsistent  or  anomalous  events  to  collection  gaps  or 
processing  errors,  rather  than  to  deception);  had  the  JN  used  the  ACH  and/or  the  ACH- 
CD  methods  of  assessment  of  the  evidence  and  the  hypotheses  that  had  been  surfaced  at 
the  various  JN  ACH-like  exercises.  Using  either  ACH  or  ACH-CD  to  review  the  JN 
planning  assumptions  against  the  available  evidence,  the  JN  planners  would  have  had 
ample  reason  to  re-examine  the  soundness  of  “Operation  MI.” 

Futhermore,  the  JN  might  have  been  able  to  overcome  the  third  major  impediment 
to  effective  counter-deception  analysis:  Failure  to  link  deception  tactics  to  deception 
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hypotheses  (failure  to  recognize  anomalous  evidence  as  indicators  of  deception).  That  is, 
the  events  and  evidence  available  to  the  JN  before  the  Battle  of  Midway  might  have  been 
assessed  as  possible  indicators  that  the  U.S.  Pacific  Fleet  was  using  denial  and  deception 
tactics  to  conceal  its  true  response  to  “Operation  MI.”  Evidence  reflecting  specific  denial 
and  deception  tactics  used  to  conceal  H2  or  H3,  and  inconsistent  with  HI,  are  shown  in 
Tables  2  and  3.  Note  that  these  indicators  included  negative  evidence  (e.g.,  No  U.S. 
aircraft  carriers  sightings  in  South  Pacific  after  1 7  May  1942;  No  apparent  objectives  for 
U.S.  carriers  in  South  Pacific,  late  May  1942). 


Table  2:  Counter-Denial  Indicators  Available  to  JN  Before  the  Battle  of  Midway 

Denial 

Tactics 

Features  of  Denial  Tactics 

General  Indicators 
of  Denial  Tactics 

Indicators  from 

Battle  of  Midway, 
May-June  1942 

Masking 

Hide  &  conceal  key 
characteristics,  while  matching 
another;  eliminate  characteristic 
patterns,  blend  characteristics  with 
background  patterns 

—Key  components 
missing,  incomplete, 
or  unaccounted; 

-High  information 
value  components 
unobserved  where 
expected 

-  U.S.  aircraft  carriers 
undetected  in  North  Pacific 
vicinity  Hawaii  or  Midway, 
until  1-3  June  1942 

-  Aerial  surveillance  aircraft 
vicinity  Midway  all  shot 
down 

Repackaging 

Add  and  change  key 
characteristics;  modify 
characteristic  patterns,  match  an 
alternative  component’s 
characteristic  pattern 

—Excessive, 
inconsistent,  or 
unexpected 
alternative 

components  detected; 
—Too  many  of  the 
wrong  things 

-  U.S.  aircraft  carriers  radio 
traffic  prevalent  in  South 
Pacific 

-  Operation  K  thwarted 

-  Midway  defenses  & 
reconnaissance  greatly 
enhanced,  3-4  Jim  1942 

-  Aleutian,  other  N.  Pacific 
defenses  not  enhanced 

Dazzling 

Obscure  key  characteristics, 
saturate  perception  by  adding 
over-powering  characteristics;  blur 
characteristic  patterns  to  increase 
observer  uncertainty 

-Unexpected 
perceptual  stimuli; 
—Atypical  or 
uncommon  patterns; 
—Unusual  intensity, 
density,  frequency 

-  Apparent  U.S.  carrier 
losses,  e.g..  Battle  of  Coral 
Sea 

-  No  apparent  objectives  for 
U.S.  carriers  in  South 

Pacific,  late  May  1942. 

Red 

Flagging 

Display  key  characteristics 
ostentatiously,  make  high 
information  value  patterns 
conspicuously  obvious,  “wave  a 
red  flag;”  generate  observer 
suspicions 

-Some,  but  not  all, 
expected  key 
components  on 
obvious  display; 
—Significant  key 
components  missing 
or  unaccounted  for 

-  U.S.  submarines  and  scout 
aircraft  at  Midway  deployed 
beyond  normal  operational 
limits,  1-3  June  1942. 

-  U.S.  aircraft  carriers 
undetected  in  North  Pacific 
vicinity  Hawaii  or  Midway, 
until  1-3  June  1942 
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Table  3:  Counter-Deception  Indicators  Available  to  JN  Before  the  Battle  of  Midway 


Deception 

Tactics 

Features  of  Deception  Tactics 

Indicators  of  Deception 
Tactics 

Indicators  from 
Battle  of  Midway, 
May- June  1942 

Mimicking 

Recreate  or  imitate  a  familiar 
characteristic  patterns;  copy 
alternative  characteristics;  create 
fictitious  entities 

—Observations 
inconsistent  with 
expected  numbers, 
patterns,  configurations 
—Insufficient  fidelity, 
inexplicable  anomalies 
—Too  many  of  the 
wrong  thing 

-  U.S.  aircraft  carriers 
radio  traffic  prevalent 
in  South  Pacific 

-  No  U.S.  aircraft 
carriers  sightings  in 
South  Pacific  after  17 
May  1942 

-  No  apparent 
objectives  for  U.S. 
carriers  in  South 

Pacific,  late  May  1942. 

Inventing 

Create  new  characteristic  patterns  with 
high  information  value;  synthesize 
realistic  indicators;  invent  key 
components 

-Insufficient  history, 
resolution,  fidelity 
-Multi-dimensional 
“thinness” 

-Inappropriate 
consistencies 
—Exploitation  of 
expectations, 
conditioning,  reflexive 
control 

-  U.S.  aircraft  carriers 
radio  traffic  prevalent 
in  South  Pacific 

-  No  U.S.  aircraft 
carriers  sightings  in 
South  Pacific  after  17 
May  1942 

-  “Midway  short  on 
water”— ”AF”  short  on 
water 

Decoying 

Create  parallel  characteristic  patterns 
forming  immaterial  entities  or 
indicators;  provide  realistic 
characteristic  patterns  to  increase 
observer  certainty 

-Insufficient  history  or 
contiguity 
-Configuration  & 
correlation  anomalies 
-Multi-spectral 
anomalies  or  resolution 
“thinness” 
—Inconsistencies  in 
spectral  or  dimensional 
resolution 

-  No  apparent 
objectives  for  U.S. 
carriers  in  South 

Pacific,  late  May  1942. 

-  U.S.  aircraft  carriers 
undetected  in  North 
Pacific  vicinity  Hawaii 
or  Midway,  until  1-3 
June  1942 

Double 

Play 

Weakly  &  suspiciously  suggest  correct 
interpretation  to  reinforce  incorrect 
interpretation;  maintain  or  display  real 
but  suspicious  characteristics  to 
decrease  observer  acceptance 

—Inconsistent  history  or 
timing  of  discrediting 
information 

—Discontinuous  volume 
or  intensity  of 
disconfirming 
information 
—Inconsistent 
selectivity  of 
information 
—Artificial  consistency 
or  uniformity  of 
discrediting  information 

-  U.S.  radio  traffic  in 
North  Pacific  vicinity 
Hawaii,  1-3  June  1942 

We  put  this  evidence  of  possible  deception  into  a  Bayesian  belief  network  and 
linked  the  evidence  to  the  relevant  events  (e.g.,  U.S.  CVs  in  SW  Pacific,  US  CVs  in  NW 
Pacific,  Midway  Island  defenses  enhanced,  see  Figure  4).  We  set  the  prior  probabilities  at 
Hl=  70%,  H2  =  20%,  H3  =  9%,  and  H4  (Other)  =  1%.  We  linked  the  events  and 
hypotheses  to  the  evidence  using  the  ACH-CD  procedures  described  above,  using 
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intermediate  probabilities  representative  of  a  counter-deception  analyst’s  expectation  of 
events  (e.g.,  US  CVs  in  SW  Pacific)  and  associated  intelligence  evidence  items  (US  CVs 
SIGINT),  given  those  events  (True  and  False)  and  those  COAs  (Intercepted,  Not 
Intercepted).  The  likelihoods  connecting  the  intelligence  evidence  to  the  relevant  events 
and  to  the  hypotheses  reflects  the  possibility  of  American  denial  and  deception.  That  is, 
the  JN  counter-deception  analyst  should  reason  that,  if  H3  were  TRUE,  but  event  US  CVs 
in  SW  Pacific  is  FALSE,  US  CVs  SIGINT  INTERCEPTED  would  very  likely  (p=.8), 
because  such  American  radio  deception  would  be  consistent  with  H3. 

The  evidence  indicating  possible  denial  and  deception  tactics  is  instantiated,  as 
shown  in  Figure  5  by  the  gray  boxes.  For  example,  SIGINT  intercepts  indicated  the  US 
aircraft  carriers  (CVs)  in  the  Southwest  Pacific  (SW  Pac).  JN  ISR  sighted  U.S.  CVs  in 
SW  Pac  on  17  May,  but  not  after.  Midway  Island  aerial  and  submarine  reconnaissance 
ranges  were  increased  from  500  miles  to  700  miles  (MI_Recce_Expanded).  The  impact 
of  these  denial  and  deception  indicators  on  the  hypotheses  reflecting  the  enemy  COAs  is 
dramatic.  In  Figure  5,  the  probability  of  HI  drops  from  70%  to  about  14%,  while  H3 
jumps  from  9%  to  over  85%. 

The  role  of  H4:  Other 

H4:  Other  in  these  models  provides  an  index  of  noise  versus  signal  in  the 
interpretation  of  the  intelligence.  All  the  intelligence  relative  to  H4  is  coded  as  />(Ezj  H4) 
=p(Ei  \  not  H4)  =  .50.  That  is,  since  the  evidence  cannot  discriminate  between  H4  and  its 
opposite  not  H4,  the  likelihood  of  H4  represents  a  baseline  of  complete  ignorance  relative 
to  the  other  hypotheses  HI — H3.  When  all  the  intelligence  is  noise,  the  likelihood  of  H4 
approaches  1.0.  In  our  models,  with  the  benefit  of  hindsight,  all  the  intelligence  evidence 
is  “pure  signal”  and  can  be  interpreted  to  have  discriminatory  significance  for  the  HI  - 
H3  hypotheses.  Had  more  noisy  intelligence  been  added  to  these  models,  the  strength  of 
H4  would  increase,  but  the  likelihoods  of  HI — H3  relative  to  each  other  would  remain  in 
the  same  relative  proportions.  That  is,  the  models  would  still  reflect  that  the  “pure 
signals”  would  indicate  the  possibility  of  American  denial  and  deception  tactics  and  the 
possibility  of  H2  and  H3  as  well  as  HI. 

While  the  effect  shown  in  Figure  5  is  powerful,  it  merely  shows  the  impact  of 
isolating  the  intelligence  evidence  that  was  most  indicative  of  possible  American  denial 
and  deception  tactics  and  then  determining  how  that  evidence  could  impact  beliefs  in 
possible  enemy  COAs.  That  is,  the  model  shown  in  Figures  4  and  5  might  have  been  used 
by  a  JN  counter-deception  analyst  to  make  the  case  that  the  success  of  Operation  MI  was 
highly  sensitive  to  indicators  that  JN  intelligence  had  noted  of  possible  American  denial 
and  deception.  Such  indicators  strongly  support  the  possibility  of  CO  A  H3,  an  American 
ambush.  Although  JN  intelligence  had  observed  the  indicators  of  American  denial  and 
deception  tactics,  the  Japanese  lacked  the  framework  and  business  process  that  allows 
these  indicators  to  be  linked  to  hypotheses  of  deceptive  COAs. 

In  making  an  overall  assessment  of  the  JN  intelligence,  all  key  intelligence  items 
should  be  weighed  along  with  these  denial  and  deception  indicators,  in  keeping  with  the 
ACH-CD  procedure.  Bayesian  belief  networks  based  on  all  the  events,  intelligence,  and 
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evidence  available  to  the  JN  prior  to  the  battle  (Appendix  1)  is  shown  in  Figure  6,  with 
the  linking  probabilities  set  using  our  ACH-CD  process,  with  the  prior  probabilities  for 
HI  through  H4  as  before.  When  all  these  evidence  items  are  instantiated,  as  they  might 
have  been  on  the  eve  of  the  battle,  the  probabilities  for  the  enemy  (U.S.  Pacific  Fleet) 
COAs  change  dramatically  (Figure  7,  Table  4). 


Table  4:  Probabilities  for  Enemy  Courses  of  Action  before  and  after  evidence  available  to  JN  before 
the  Battle  of  Midway  is  considered  using  ACH-CD  process. 

Enemy  (U.S.  Pacific  Fleet)  Courses  of  Action 

Notional  Prior 
Probability 
[Figure  6] 

Probability  after 
evidence  is 
considered 
[Figure  7] 

HI:  U.S.  Pacific  Fleet  would  be  surprised,  and  would 
respond  to  the  JN  invasion  of  Midway,  sending  its 
remaining  aircraft  carriers  to  attempt  to  retake  Midway. 

70% 

<2% 

H2 :  U.S.  Pacific  Fleet  would  be  surprised,  and  would  not 
respond  to  the  JN  invasion  of  Midway;  letting  Japan 
extend  its  naval  base  perimeter  to  mid-Pacific. 

20% 

<1% 

H3:  U.S.  Pacific  Fleet  would  not  be  surprised,  and  its 
carriers  will  be  waiting  near  Midway  to  attack  the  JN 

Carrier  Battle  Group  (Kido  Butai). 

9% 

98% 

H4:  Other  COA 

1% 

<1% 

Additionally,  if  the  evidence  available  to  the  JN  in  April,  May,  and  through  3 
June  1942  are  instantiated  in  the  Bayesian  belief  network,  the  changing  probabilities 
(Table  5)  reflect  how  the  accumulating  intelligence  might  have  shifted  JN  beliefs  in  the 
various  enemy  COAs  in  response  to  “Operation  MI.” 


Table  5:  Probabilities  for  Enemy  Courses  of  Action  based  on  evidence  available  to  JN  in  April,  May, 
and  up  to  3  June  1942. 

Enemy  (U.S.  Pacific  Fleet)  Courses  of  Action 

April  1942 

May  1942 

3  June  1942 

HI:  U.S.  Pacific  Fleet  would  be  surprised,  and  would 
respond  to  the  JN  invasion  of  Midway,  sending  its 
remaining  carriers  to  attempt  to  retake  Midway. 

69% 

29% 

<2% 

H2 :  U.S.  Pacific  Fleet  would  be  surprised,  and  would  not 
respond  to  the  JN  invasion  of  Midway;  letting  Japan 
extend  its  naval  base  perimeter  to  mid-Pacific. 

20% 

1% 

<1% 

H3:  U.S.  Pacific  Fleet  would  not  be  surprised,  and  its 
carriers  will  be  waiting  near  Midway  to  attack  the  JN 

Carrier  Battle  Group  (Kido  Butai). 

10% 

70% 

98% 

H4:  Other  COA 

1% 

<1% 

<1% 

The  pattern  in  Table  5,  the  possible  JN  appreciation  of  alternative  U.S.  Navy 
COAs,  is  symmetrical  with  the  growing  understanding  of  Japanese  plans  and  intentions 
by  the  Pacific  Fleet  intelligence  officers  and  commanders  in  Hawaii.  By  the  end  on  May 
1942,  they  had  pieced  together  90%  of  the  plans  for  “Operation  MI,”  had  successfully 
portrayed  the  remaining  US  carriers  as  being  in  the  Southwest  Pacific  through  radio 
deception,  and  had  planned  their  ambush  for  the  Japanese  carriers.  When  the  Kido  Butai 
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arrived  at  Midway,  the  rest,  as  the  saying  goes,  is  history.  Had  the  Japanese  had  better 
counter-deception  business  processes,  their  situation  assessment  might  not  have  been  so 
far  inferior  to  that  of  their  opponent,  and  the  outcome  might  not  have  been  so  one-sided. 

Taken  as  a  whole,  both  the  ACH  and  ACH-CD  techniques,  which  showed  that  H2 
and  H3  could  not  be  ruled  out  as  American  CO  As,  and  that  HI,  the  enemy  CO  A  that  the 
JN  used  as  the  basis  for  Operation  MI  was  merely  possible,  not  highly  probable,  as  the  JN 
planners  believed.  When  denial  and  deception  indicators  are  identified  and  considered  in 
the  Bayesian  belief  network,  the  impact  of  this  intelligence  on  the  probability  of  H3  is 
dramatic,  and  the  possibility  of  an  American  deception  and  ambush  becomes  extremely 
difficult  to  ignore.  Using  a  counter-deception  business  process  to  evaluate  the  sensitivity 
of  possible  enemy  COAs  to  all  the  available  evidence  could  have  aided  the  JN  planners  to 
track  the  American  deception  and  perhaps  to  avoid  the  utter  debacle  of  “Operation  MI.” 

In  summary,  the  JN  operational  planners  and  intelligence  analysts  might  have 
used  Heuer’s  ACH,  or  our  ACH-CD  process,  to  review  the  available  evidence  and  the 
planning  assumptions  underlying  “Operation  MI,”  as  well  as  the  other  COA  hypotheses 
that  were  surfaced  in  the  JN  ACH-like  exercises.  Using  the  ACH  technique,  they  might 
have  noted  that  the  evidence  available  before  the  Battle  of  Midway  was  just  as  consistent 
with  H3,  the  deception  hypothesis  (the  U.S.  Pacific  Fleet  would  ambush  the  JN  attack)  as 
it  was  with  HI,  the  hypothesis  on  which  “Operation  MI”  was  based  (that  the  Pacific  Fleet 
would  be  surprised  by  Operation  MI  and  would  respond  on  the  JN  time-table). 

Furthermore,  had  the  JN  compared  this  evidence  to  general  indicators  of  denial 
and  deception  tactics,  they  might  have  noted  further  support  for  the  hypothesis  that  the 
U.S.  Navy  would  not  be  surprised  by  Operation  MI  and  was  using  denial  and  deception 
to  cover  its  riposte. 

Had  the  JN  planners  used  tools  such  as  we  developed  to  support  a  counter¬ 
deception  business  process,  they  would  have  been  able  to  isolate  those  items  of  evidence 
that  were  most  significant  in  supporting  the  various  possible  U.S.  COAs.  These 
sensitivities  can  reinforce  ISR  operations  and  counter-planning;  e.g.,  could  have  aided  the 
JN  in  the  design  of  planned  or  natural  operational-intelligence  “experiments,”  as 
recommended  by  R.  V.  Jones’s  “theory  of  spook  unmasking,”  to  force  the  U.S.  Pacific 
Fleet  to  reveal  more  evidence  of  its  intentions  and  dispositions.  For  example,  a  realistic 
JN  feint  in  the  Coral  Sea  towards  New  Guinea  or  Australia  in  late  May  1943  might  have 
forced  the  U.S.  Navy  to  react  and  thus  uncovered  Nimitz’s  radio  deception. 

CONCLUSIONS  AND  AREAS  FOR  FURTHER  RESEARCH 

We  have  demonstrated  how  a  counter-deception  business  process  based  on  ACH- 
CD  can  be  applied  to  military  counter-deception.  We  showed  how  evidence  available  to 
the  Japanese  Navy  prior  to  the  Battle  of  Midway,  if  analyzed  using  ACH-CD,  might  have 
aided  detection  of  the  American  deception  that  allowed  the  U.S.  Pacific  Fleet  carriers  to 
surprise  and  ambush  the  Japanese  carriers  threatening  Midway  Island. 
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We  have  developed  a  software  system  that  automates  ACH-CD.  Our  effort  now  is 
on  extending  the  core  ACH-CD  process  to  deal  with  the  issue  of  “local  versus  global 
deception,”  and  the  generation  of  more  complex  courses  of  action.  We  are  reviewing 
ongoing  research  on  algorithms  that  detect  anomalies  to  incorporate  in  our  counter¬ 
deception  business  process  and  tools  (e.g.,  Dragoni  1996,  Johnson  2004,  Santos  2004, 
Sarteretal.  1997]. 

We  are  building  an  interface  to  our  deception  planning  system  that  will  help  users 
easily  create  realistic  domain  descriptions.  The  deception  planning  system  will  fill  in 
these  plans  and  create  CO  As  as  alternatives.  On  the  back  end,  the  planning  system 
suggests  deception  tactics  to  keep  an  adversary  from  recognizing  the  true  plan 
(dissimulation)  and  ways  to  give  the  adversary  a  false  apprehension  of  reality 
(simulation).  A  temporal  model  generated  with  the  alternate  courses  of  action  will  be  an 
important  input  to  this  process.  The  deception  planning  system  will  be  extended  to 
counter-deception  planning  using  AP’s  counter-planning  process. 

We  have  been  conducting  experiments  to  assess  how  well  our  counter-deception 
business  process  can  reliably  (a)  plan  deceptions,  and  (b)  detect  deceptions.  As  indicated 
above,  the  system  has  been  successful  in  determining  previously  unknown  details  of  how 
historic  deceptions  have  succeeded,  such  as  the  Battle  of  Midway,  and  in  reflecting  how 
better  assessment  of  intelligence  using  a  counter-deception  business  process  can  increase 
the  likelihood  of  detecting  and  characterizing  deceptions. 
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Appendix  1 

Events,  Intelligence,  And  Evidence  Obtained  by  JN  Before  the  Battle  of  Midwa; 


Date 

Events  and  evidence 

Jan-Mar  1942 

U.S.  carrier  raids  against  the  Marshalls,  Rabaul,  Wake,  Marcus  Island,  and  eastern  New 
Guinea:  Indicates  ineffectiveness  of  JN  reconnaissance  and  warning  intelligence. 

10  Mar  1942 

U.S.  carrier  aircraft  surprise  ADM  Kajioka’s  Japanese  Landing  Forces  at  Lae  and 
Salamaua  in  the  Solomon  Sea:  U.S.  carriers  able  to  position  themselves  to  ambush  the 

JN  landing  forces  and  escape.  Indicates  effectiveness  of  U.S.  intelligence  and 
reconnaissance. 

2-5  Apr  1942 

Naval  General  Staff  and  Combined  Fleet  Planning  Debate:  JN  war-gamers  assumed 

U.S.  forces  would  conform  to  JN  plan  and  timetable  (HI).  No  efforts  to  war-game  a 
delayed  U.S.  response,  after  Japanese  fleet  had  departed  (H2).  No  efforts  to  war-game 
U.S.  surprise  at  Midway  (H3).  No  JN  estimates  of  possible  impacts  of  a  U.S.  victory  (or 
draw)  at  Midway  on  U.S.  or  Japanese  strategy,  operations,  or  morale.  Indicates  narrow 
assumptions  governing  JN  planning. 

5-9  Apr  1942 

Kido  Butai  Surprised  in  Indian  Ocean:  Detected  by  land-based  reconnaissance  and 
bomber  aircraft,  and  surprised  by  British  cruisers  DORSETSHIRE  and  CORNWALL, 
carrier  HERMES  and  destroyer  VAMPIRE.  Indicates  weak  JN  carrier-based  counter¬ 
surveillance,  intelligence,  and  reconnaissance. 

18  Apr  1942 

Sharp  increase  in  U.S.  Navy  radio  communications  near  homewaters:  JN  traffic  analysis 
correctly  indicate  Doolittle  Raid. 

1-6  May  1942 

Combined  Fleet  War  Games:  raise  the  contingency  that  U.S.  carrier  task  force  might 
appear  on  Kido  Butai  flank  during  scheduled  air  attack  on  Midway:  JN  umpire  negates 
effective  contingency  response  planning. 

31  May  -  3  Jun 
1942 

U.S.  units  occupying  French  Frigate  Shoals:  Operation  K  forestalled  (plan  for  JN  subs 
to  rendezvous  at  French  Frigate  Shoals  to  refuel  seaplanes  flying  from  Wotje,  to 
reconnoiter  Pearl  Harbor) 

7-9  May  1942 

JN  SIGINT:  detected  U.S.  carrier  force  in  the  Coral  Sea  prior  to  Operation  MO-Battle 
of  the  Coral  Sea. 

7-9  May  1942 

JN  report  both  U.S.  carriers  (LEXINGTON  and  YORKTOWN)  sunk  in  Battle  of  the 
Coral  Sea:  JN  Naval  Staff  assumes  U.S.  has  two  remaining  carriers  (HORNET  and 
ENTERPRISE). 

15-16  May 

1942 

JN  air  reconnaissance:  Identifies  HORNET  and  ENTERPRISE  in  Solomons  Islands. 

May-3  Jun  1942 

JN  COMINT:  Identifies  HORNET  and  ENTERPRISE  radio  communications  in  South 
Pacific.  JA  Naval  Staff  concludes  U.S.  has  not  detected  JN  Midway  intentions 
(Operation  MI). 

18-20  May 

1942 

JN  COMINT:  Reports  Midway  radio  indicates  island  short  on  water,  Hawaii  will  re¬ 
supply. 

May  1942 

JN  Air  Reconnaissance:  All  long-range  Japanese  aerial  reconnaissance  missions  to 
Midway  are  destroyed. 

24  May  1942 

JN  Combined  Fleet  Estimate:  At  final  Table-top  Maneuvers  ADM  Ugaki  (Yamamoto’s 
Chief  of  Staff)  states:  “It  is  hard  to  make  accurate  judgment  of  the  next  enemy 
move. .  .but  according  to  newspapers  they  were  reported  to  be  heading  for  Australia.  At 
present,  the  whereabouts  of  two  enemy  carriers  is  unknown — either  in  Australia  or 
Hawaii.”  Failure  to  include  all  possible  hypotheses  (e.g.,  H3). 

24  May  1942 

JN  Combined  Fleet  Estimate:  JN  intelligence  on  Aleutians  was  abysmal,  out-of-date, 
vastly  over-estimating  U.S.  ground  forces,  under-estimating  naval  forces,  and 
completely  ignorant  of  land-based  air  forces. 

Midway  Revisited 


©November  2004  The  MITRE  Corporation 


23 


Events,  Intelligence,  And  Evidence  Obtained  by  JN  Before  the  Battle  of  Midway _ 

Date _ Events  and  evidence _ 

29-31  May  JN  Reports:  U.S.  reconnaissance  arc  extended  from  500  to  700  miles  from  Midway. 

1942  U.S.  submarine  transmission  indicated  JN  Transport  Group  had  been  discovered  west  of 

Midway.  U.S.  Pacific  Fleet  radio  traffic  and  ratio  of  urgent  messages  greatly  increased 
in  Hawaiian  and  Alaskan  waters:  JN  indications  of  U.S.  readiness  and  preparedness  for 
_ Operation  MI. _ 

2  Jun  1942  Kido  Butai  failed  to  receive  urgent  transmission:  JN  Naval  Staff  estimates  that 

Americans  had  discovered  Midway  operation  and  might  be  sending  carriers  to  ambush 
_ Kido  Butai. _ 

3  Jun  1942  Kido  Butai  estimate:  “It  is  not  believed  that  the  enemy  has  any  powerful  unit,  with 

_ carriers  as  its  nucleus,  in  the  vicinity/' _ 

3  Jun  1942  U.S.  Midway-based  PBY  scout  aircraft:  Spot  JN  Invasion  Force  where  predicted  (24 

_ hours  before  Japanese  expected  to  be  detected). _ 

0820  4  Jun  1942  JN  Scout  aircraft:  Reports  U.S.  carriers  within  150  miles  of  Kido  Butai . _ 

1024  4  Jun  1942  U.S.  carrier  dive  bombers  sink  three  Kido  Butai  carriers  within  minutes,  later  sink 

_ fourth. _ _ _ 

Primary  Sources:  Henry  F.  Schorreck,  Department  of  the  Navy,  Naval  Historical  Center,  Battle  of 
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Appendix  2 


Evaluating  JN  Hypotheses  supported  tr 

V  Evidence  (likelihood  of  hypothesis  given  evidence):  pfHilE,)  1 

Hypotheses 

HI:  U.S.  will 

H2 :  U.S.  will  not 

H3:  U.S.  will  be 

respond  to  JN 

respond  to  JN 

waiting  near 

Evidence 

invasion  of  Midway 

invasion  of  Midway 

Midway 

U.S.  carrier  raids:  Indicates 
ineffectiveness  of  JN  ISR. 

N 

Y 

Y 

U.S.  carriers  surprise  Landing  Forces: 
Indicates  effectiveness  of  U.S.  ISR. 

N 

N 

Y 

Kido  Butai  Surprised  in  Indian  Ocean: 
Indicates  ineffectiveness  of  JN  ISR. 

N 

Y 

Y 

JN  traffic  analysis  correctly  indicates 
Doolittle  Raid. 

Y 

Y 

N 

U.S.  units  occupying  French  Frigate 
Shoals:  Operation  K  forestalled 

N 

? 

Y 

JN  SIGINT:  detected  U.S.  carrier  force 
in  Coral  Sea 

Y 

N 

N 

JN  Naval  Staff  assumes  U.S.  has  two 
remaining  carriers 

Y 

Y 

N 

JN  air  reconnaissance:  two  U.S.  carriers 
in  Solomons  Islands. 

Y 

? 

N 

JN  COMINT:  two  U.S.  carriers  in 
Solomons  Islands. 

Y 

? 

N 

JN  COMINT:  Midway  short  on  water, 
Hawaii  will  re-supply. 

Y 

N 

? 

All  Japanese  aerial  reconnaissance  to 
Midway  destroyed. 

N 

? 

Y 

JN  Combined  Fleet  Estimate:  “two 
enemy  carriers — either  in  Australia  or 
Hawaii.” 

Y 

Y 

N 

JN  intelligence  on  Aleutians  out-of-date 
&  inaccurate 

Y 

Y 

Y 

JN  indications  of  U.S.  readiness  and 
preparedness  for  Operation  MI. 

N 

N 

Y 

JN  Naval  Staff  estimates:  Americans 
had  discovered  Midway  operation, 
might  ambush  Kido  Butai . 

N 

N 

Y 

Kido  Butai  estimate:  no  enemy  carriers 
in  vicinity. 

Y 

Y 

N 

U.S.  Spot  JN  Invasion  Force  where 
predicted 

N 

? 

Y 

U.S.  carriers  within  150  miles  of  Kido 
Butai. 

N 

N 

Y 

Y  9 

Y  7 

Y  10 

Totals 

N  9 

N  6 

N  7 

?  0 

?  5 

?  1 
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Appendix  3 


Evaluating  JN  Evidence  and  Hypotheses  (likelihood  of  evidence  given  hypothesis-True  and 
hypothesis-False):  p(Ej|Hj)  &  />(E||~Hj) 

Hypotheses 

HI:  U.S.  will 

H2:  U.S.  will  not 

H3:  U.S.  will  be 

respond  to  JN 

respond  to  JN 

waiting  near 

Evidence 

invasion  of  Midway 

invasion  of  Midway 

Midway 

U.S.  carrier  raids:  Indicates 
ineffectiveness  of  JN  ISR. 

N 

N 

Y  1 

U.S.  carriers  surprise  Landing  Forces: 
Indicates  effectiveness  of  U.S.  ISR. 

N 

N 

Y 

Kido  Butai  Surprised  in  Indian  Ocean: 
Indicates  ineffectiveness  of  JN  ISR. 

N 

N 

Y 

JN  traffic  analysis  correctly  indicate 
Doolittle  Raid. 

Y 

Y 

N 

U.S.  units  occupying  French  Frigate 
Shoals:  Operation  K  forestalled 

N 

Y 

Y 

JN  SIGINT:  detected  U.S.  carrier  force 
in  Coral  Sea 

Y 

Y 

N 

JN  Naval  Staff  assumes  U.S.  has  two 
remaining  carriers 

Y 

Y 

N 

JN  air  reconnaissance:  two  U.S.  carriers 
in  Solomons  Islands. 

Y 

Y 

N 

JN  COMINT:  two  U.S.  carriers  in 
Solomons  Islands. 

Y 

Y 

N 

JN  COMINT:  Midway  short  on  water, 
Hawaii  will  re-supply. 

Y 

Y 

? 

All  Japanese  aerial  reconnaissance  to 
Midway  destroyed. 

N 

N 

Y 

JN  Combined  Fleet  Estimate:  “two 
enemy  carriers — either  in  Australia  or 
Hawaii.” 

Y 

Y 

N 

JN  intelligence  on  Aleutians  out-of-date 
&  inaccurate 

? 

? 

? 

JN  indications  of  U.S.  readiness  and 
preparedness  for  Operation  MI. 

N 

N 

Y 

JN  Naval  Staff  estimates:  Americans 
had  discovered  Midway  operation, 
might  ambush  Kido  Butai. 

N 

N 

Y 

Kido  Butai  estimate:  no  enemy  carriers 
in  vicinity. 

Y 

Y 

N 

U.S.  Spot  JN  Invasion  Force  where 
predicted 

N 

N 

Y 

U.S.  carriers  within  150  miles  of  Kido 
Butai. 

N 

N 

Y 

Y  8 

Y  9 

Y  9 

Totals 

N  9 

N  8 

N  7 

?  1 

?  1 

?  2 
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ENDNOTES 


I  The  idea  that  Krypton  release  might  be  used  for  such  a  nuclear  bluff  has  been  suggested  by,  among  others, 
Carnegie  Endowment  for  International  Peace  analyst  James  Wolfsthal;  c.f.,  Merrill  Goozner,  "Nuclear 
Blackmail:  The  growing  North  Korean  threat  --  and  why  containment  may  still  be  our  best  means  of 
staving  it  off.”  The  American  Prospect ,  August  8,  2003.  http://www.prospect.org/print- 
fnendly/webfeatures/2003/08/goozner-m-08-08.html  On  Krypton-85  as  a  nuclear  enrichment  indicator,  see 
Office  of  Technology  Assessment,  Technologies  Underlying  Weapons  of  Mass  Destruction.  OTA-BP-ISC- 
115,  December  1993;  David  E.  Sanger  and  Thom  Shanker,  “North  Korea  Hides  New  Nuclear  Site, 
Evidence  Suggests,”  New  York  Times ,  July  20,  2003;  George  Wehrffitz  and  Richard  Wolffe,  “How  North 
Korea  Got  The  Bomb,”  Newsweek  (International  Edition),  October  27, 2003;  and  interview  with  David 
Albright,  president  of  the  Institute  for  Science  and  International  Security,  Washington,  DC.  CNN  Aired 
May  11,  2003  -  17:00  ET  http://cnnstudentnewsxnn.eom/TRANSCRIPTS/0305/l  l/nac.00.html 

II  The  Bayesian  ACH  BBN  is  more  conservative  in  other  directions  as  well.  That  is,  if  the  facts  are  that 
there  is  no  nuclear  enrichment,  and  there  is  pipeline  testing,  the  observation  of  Krypton  is  less  likely  to 
suggest  deception  to  Bayesian  reasoners.  In  the  non-Bayesian  ACH  BBN,  no  nuclear  program  and  pipeline 
testing  yields  7:3  odds  in  favor  of  the  deception  hypothesis;  while  the  Bayesian  ACH  BBN  reflects  the 
more  conservative  5:5  odds  for  deception;  that  is,  die  Bayesian  conclusion  is  that  Krypton  could  be  due  to 
deception  and  the  testing,  or  to  just  the  testing  alone.  Similarly,  when  pipeline  testing  can  be  ruled  out,  the 
non-Bayesian  ACH  BBN  yields  7:3  odds  in  favor  of  enrichment  and  4  to  1  odds  against  deception;  while 
Bayesian  ACH  BBN  conservatively  considers  the  same  evidence  as  reflecting  6:4  odds  for  enrichment  and 
4:6  odds  for  deception. 
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